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AjESTPACT 

Factor loadings, used directly or as the basis of 
binary values, are not appropriate as weights to produce component 
scores from a rotated solution* A series of examples showing the 
results of an incorrect measurement of components is given. Several 
correlation matrices were .taken from books on factor analysis and 
multivariate analysis* Each matrijc was submitted to a principal 
components decomposition, and those vectors whose roots were greater 
x.han 1*0 were then rotated according to the normalized varimax 
criterion. Following this^ several matrices were calculated. A clear 
pattern resulted, components supposedly orthogonal to each other 
often come out highly correlated* In addition, factor loadings that 
are small in ,the rotated principal components structure are often 
moderate to large in R/zh and R/zk. Thus, the researcher who 
interprets and names components finds that his scores have a 
completely different meaning than he anticipates* Any use of these 
scores in further analysis will result in a serious distortion of 
conclusions* (Author/KM) 
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Introduction 

Factor analysis Has proved to be a popular mode of data analysis in 
the behavioral sciences. Hovaver, what is often referred to as factor 
analysis is actually based on the component model rather than the factor 
model and should be called component analysii.- Certainly, of all the 
methods currently available, the most widely used is the principal compo- 
nents analysis folloved by the varinax rotation, it remains popuLar not 
because it is based on a Btthod or nodel which is universally appropriate, 
but because it is simple, beeause aomputer programs are readily available 
and because component scores are eagil^ calculated, Yet, in Bplte of the 
fact that component scores are easily found, there is a history of them 
being found incorrectly. 

Good discuss ions of the Beasurement of components from the rotated 
principal components solution have been written by such authors as 
Harman (I967), Kaiser (1962), and Glass and Maguire (1966). Still, many 
users of the model have their own ideas of how theee acores should be ob- 
tained. Their Biethods have great intuitive aHJeal aad appear reasonable 
and obvious. One of the most popular of these methods Uies factor loadings 
as weights of the standardized variables and the resulting linear combinatloni 
are called "factor scores." Another method, claimed to yield approximate 
factor scores, sums those standapdized wriables with "high" loadings on a 
pf-rticular oomponent. Thus, weights of zero and one are uBed in the linear 
eoa^site , 

Both methods are techoically incorrect. However, a detailed discusBion 
of the mathtMtlcal reasons why the methods are incorrect may not be 
persuasive to the user who finds them Intuitively appealing. Yet these users 
need to be convinced. For this reaBon, the theoretical development of this 
study is supplemented by a series of examples showing the results of an 
inco*rect measurement of components. 
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Theoretical development 

If n variables are observed for each of N iubjects, the component aodel, 
stated in matrix form, is given by 

T T ' 

where Z is the n-.N matrix of standardized scores, A„ is the complete nxn 
pattern matrix and is a nxN matrix of component scores. If repreientB 
a complete principal oomponenta solution, then 

However, the user of the component model usually wishes to retain only the 
.largest components, say the first m. Thus wt work with only the first a 
columns of A^ and the first m rows of F^, which we might denote as A and F 
respectively. Since A is ar; nxm matrix and has no inverse, we cannot use 
equation (2) to solve for coniponent acores. Still, & simple solution for 
P exists; 

(3) F = (A'A)"^ a'z m b"^ A'Z , 

2 

where A'A ^ D is a diagonal matrix of the n largest eigenvalues of the 
correlation matrix. It is readily proved that the eovarlance matrix. S , 
of the components equals the Identity matrix and the matrix of variable by 
component eorrelations, E^j,, equali the pattern matrix A. Alio, since the 
inverted eigenvalues scale the rows of A', the factor loadings become the 
effective weights in the linear eombinations which produce the component 
scores. Thus, the first method ffltntloned in the Introduction Is oorrect 
but for a scaling constant and the second niethod, using weight s of zero and 
one, becomes an approximation which Schwelher (1967) has argued 
some deiirable properties. 

However, if an orthogonal rotation is performed which results in a new 
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matrix of loadings, gay Bj vi now havt a new itt of coaponent scores, a, 
vhich can be ghowQ to satisfy tha following equation* 

As before, the aovarianee matrix of these component scores, denoted S i 

G 

equals the identity latrix and the matrix of TOriable by component eorrelatiDn 
R^gi equals the rotated matrix of loadingg B* 

A cospariaon of equatlani (3) and (U) indicatei a great deal of simi- 
larity and one impertant difference* It will be observed that, while A 'A 
is a diagonal matrlxj the product B*B is not, Hence^ the weights required to 
caldulate component eeoras are no longer simply scalinga of the ksadlngs in 
the rows of B'. If wa perilst in the use of the rows of B* ai walghtlng 
veetors, we will find scores which do not reflect the properties of the 
aolution. 

We can now formalize method one relative to an orthogonaHy rotated 
principal componenti solution. Using loadings as weights, we find a set of 
composite scores , denoted H| given by 

(5) H ^ B' Z. 

The matrix H 'will not equal the correct scores 0, nor will it poiseas proper- 
ties of the solution given by B* 

Two matrices were utilized to determine the extent to which the in- 
correct component scorea do not reflect the solution* The first MtriXf R— ^ 
contains the variable by component correlations. If K were correct, 1^^^ and 
B would be equal* The second is the correlation matrix, of the incorrect 
component scores* The camponents should be uneorreiated, so ihould equal 
the identity^ natrix. 



starting vith eqaatlen (5) we can eaiily derive S , the matrix of 

H 



component covariances: 



S ^ i H H' 
H N 

1 

But N ZZ' = R 

SQ 

{6) St, » B' R B. 

n 

It oan further be derived that thii expression will not reduae do^n to a 
diagonal o^trix. 

To find we need the variances of vhioh reside in the diagonal 

of S * Denoting a diagonal matrix of these variances as D ^ m find 

H 

«ZH - J ^ 

which will not in general equal the matrix B of loadings. 
Finally, 

ApLin, should he an identity matriX| but ean be shown not to be. 

Before studying the three nmtrioei developed above, it is useful to 
conaidar ineorreot component scores of the seaond variety* If, rather than 
using elements of B as weighti, we simply sum the standariMd ieorts of thoa 
variables with "large" loadlnga, we are effeetlvely using a binary matrix of 
weights. It wat arbitrarily dealded to aislgn a wel^t of t '.^0 to nil 
loadlngi whose absolutt value greater than •J and a zero to all loadlngi 
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of .5 or less In absolute value. Denote by c thia binary analogue to B. 
Then Gomponent scores 

(9) K = C Z 

have a covariance matrix *^ 

and a correlation matrix 

where D^^ is a diagonal matrix of component seore varianeea. Algo, the 
matrix of mrlable by component correlations ig given by 

With the uatheiDftties dBveloped, it remains to study these matrices 
for each procedure. The matrices were calculAted for eaeh of a variety 
of exanples and coETiAred to the form which they would take if they were 
an adequate reflection of their reipective solutions. 

Illuatratlona and Results 

Several correlation matricea were tkken froo books on factor analysis 
and multivariate analysis. Each matrix was submitted to a principal com- 
ponents decomposition and those vectors whose roots were greater than l.o 
were then rotated according to the normalized varlnax criterion. Following 
this, matrices E^^, R^^, Bjgj and 8^^, among others, were calculated. A 
clear ^ttern resulted and five examples best illuitrating this pattern 
were ehosen and are reported In Tables "li-le below. Table If contains the 
only example in irtiich the pattern was not obvious. 7 

The pattern exhibltid is clear. Con^onents siyiosedly orthogonal to 
each other often come out highly correlated (e.g. r„ „ ^ .86 in Table Id) 



ERIC 



riluitretive Ettmple.^ of Variable- by-CQmpooent and 
Component- by-Component Correlation *l!iitritei, Uaing Correct Components 
(G), COTiponenta Using Loadings ^as' Ktights (H) and 
CompQoanti Using- Binary- Weighti (K) 



la: Eight Phyiical Variablea from Itorman (1967^ p. 8o) 



ZH 



.93 


.63 


.93 


.58 


.91 


.55 


.9Z 


.So 


.57 


. .91 


.i+8 


.8U 




.80 


.52 


.78 



1.00 

0.7U 



0,7i* 
1.00 



.90 
.93 
.9S 
.90 

.25 

.18 
.11 
.25 



.26 
.20 
.16 
.23 
.89 
.Bk 
,Bk 
.75 



1.00 
0,00 



0.00 
1.00 



.9k 


.U6 


.95 


.Ul 


.93 


. .38 


.93 


.^+3 


,kk 


.91 


.37 


.86 


.30 


.8U 


.Uo 


. .80 



^ ^ - 

1.00 o.kk 
0»kk 1.00 



Ibj light Variablei on 100 Reetangles from Cooley and Lohnes (1971, p.l3U) 





ZH 


.95 


.55 


M 


.90 


.96 


.56 


M 


.89 


.99 


...82 


.95 


.93 


.82 


.99 


.99 


.70 



HH 



1.00 
0.79 



0.79 
1.00 



^ ha 
.99 
.09 
.99 
.10 

.91 
.77 
.55 
^97 

r- % 

1.00 

0.00 



B 




r= ^ZK _ 


.06 




,^ M 


.99 




M .9U 


.56 




.9^ , .1+7 


.89 




M 


.82 




.99 .75 


.?3 




.96 .8g 


.99 




.81+ .98 


.70 




^99 .6|_ 



0.00 
1.00 



— m 
1.00 
0.7^ 



0.7U 
1.00 



Ici A 9 Variable Exai^le from Hon t (I965, p.lSS) 



.91 .21 .53 

.23 .56 

.90 .39 .53 

.27 .85 ,1+0 

.18 ,87 .26 

.25 .87 ■ .31 

.i<.5 .33 .8U 

.1+5 .25 .81+ 

.53 .1^9 .80 

1.00 o.uo 0.7T 

o.uo 1.00 0.51 

0.71 o.5r 1.00 



ZG 



^ B 



.92 
.91 
.89 
.07 
.03 
.09 

.15 
.16 
.26 



_ GG 
1.00 0.00 
0.00 1.00 
0.00 0.00 



.00 


.18 


.02 


.22 


.21 


.15 


.83 


.19 


.89 


.01+ 


,88 


.07 


.11 


.85 


.01 


.88 


.29 


.71 







0.00 
0.00 

o.op 



_ ZK 

^3 .09 

,9h .11 

.91 .27 

,18 ,86 

.11 ,89 

.17 .89 

.33 .22 

.33 .Ih 
.1+0 



.38 

.38 
.31 
.19 
.23 
.86 
.86 

.35 .8l_ 
1.00 0.17 0.1+2 

0.17 1.00 0.2a 

O.U2 0.28 1.00 



3,d: A Conti'jved 10 Variable Example from Mulaik (1972, p. 228) 



.59 

M 
.70 
.73 

.39 

-52 
.86 
.90 
.86 
.80 



.89 

.76 
.85 
.65 
.58 
.50 
.50 

.69 
.61 

.83 



1.00 0.83 0.81+ 
0.B3 1.00 0.86 
0.8U 0.86 1.00 



ha = s 





rri6 


All 
1 0*+ 


oil"" 

• 3'+ 


] .50 




! .05 


tOl 


.19 


i -37 


.58 




1 .1+1* 


.79 


.06 


1 .63 


.86 




' .1+1+ 


.19 


.71* 


i .66 


.-73 




„.03 


.35 


.75 




1.31+ 


.77 




.20 


.11 


.81 




.1+5 


.53 




.9it 


.06 


.10 




.89 


.61 




.87 


.31+ 


.10 




.91 


.75 




.76 


.12 


.1*5 




.88 


.8l_ 




.^51 


.51* 




I 

] 


.80 



I. 00 0.00 0.00 
0.00 1.00 0.00 
0.00 0.00 1,00 



ZK 

.90 

.79 
.86 

.55 
.h9 
M 



.51 
.82 

_ ^ 
1.00 0.66 
0.68 1,00 
,0,58 0.59 



.56 

.1+0 

.39 

.86 

.80 

.83 

.36 

.1+2 

.61 1 

.6J+J 



0.58 

0.59 
1.00 



lot 



A Contrived 10 Variable EMLtnple from rruchter (l95l+,p.36) 







%H 














.1+8 
















.91 


.78 




.03 


.81 


.58 




.1+8 


.80 


.89 




.02 


.62 


.78 




.33 


.3U 


.88 




-.01 


.03 


.99 




.91 


.79 


■M 




.83 


.56 


-.01 




.81 


.90 


.1+6 




.63 


.77 






.36 


.88 


.31 




.05 


.99 


-.03 




.90 


.1+6 


.81 




.79 


-.02 


.61 




.79 


.1+7 


.92 




.59 


-.01 


.81 




.87 


.32 


.36 




.99 


-.05 


,01 




.96 


.81) 


.85 




^9 


M. 


















1,00 


0.71 


0.71 






1.00 


0.00 


0.00 


0.71 


1.00 


0,69 




0.00 


1.00 


0,00 


0.71 


-.69 


1.00 




0.00 


0.00 


1.00 



.51 
.51 

.36 

.90 
.81 



.91 
.79 
.33 
.79 
.91 



.36 .89 

.90 .1+6 

.81 .1+6 

.86 .33 

.97 .8li 



.79 
.91 
.89 
M 
.1+6 
.33 
.79 
.91 
.33 
.8U 



1.00 0,72 0.72 
0,72 1.00 C.69 
0.72 0.69 1.00 



1ft Tail Variables BeajLted to Edueatlonal and Occujfttional 

Aspirations of 17 Year-Old Boys from VandeGeer (1971, p.lfis) 
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.10 


.1+5 


-.1+7 




-.16 


.1+6 


.65 


-.10 




.23 


.1*9 


.51+ 


.22 




.35 


.58 


,1+2 


.1+1+ 




.55 


.73 


.50 


-.15 




.72 


.31+ 


.15 


-.77 




.37 


,56 


.80 


.08 




.27 


.53 


.82 


.01+ 




.21 


.83 


.56 


- .13 




.81 


.81 


*56 


-.06 




.78 



x-.oo 
0.78 
-.05 



0.78 

i.oo 
-.01+ 



-.05 
-.01+ 

1.00 



1.00 
0.00 
0.00 



.58 

.61 

M 
.19 
■ .19 
-.03 
.76 
.31 
.22 
.23 



GG 
0.00 
1.00 
0.00 



B 

-.1+6 
-,08 
.21+ 
.1+7 
-.13 
-,76 
.10 
.06 
-.10 

-.03 



= I 



0.00 
0.00 
1.00 



.10 
.35 
.35 
.67 
.75 
.21 
,kk 
.1+0 
.81+ 
.81 



1.00 
0.1+5 
-.21 



ZK 

.58 
.70 

.35 
.26 

..35 
.13 ■ 
.78 
.80 
.38 
.38 

0.1+5 
1.00 

-.13 . 



-.11 

-..10 

-.09 

.01+ 
-.20 
•1.00 
-.08 

-.07 
-.28 
-.20 



-.21 
-.13 
1.00 
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In addition I faotor loadingi which are amall in matrix the rotated 

prinolpal coraponents structuri| are often moderate to Large in R_ and 

^ ZH 

^ 'IS ^hila g = .61 and r^ m ,51 in ^ble Id). Thus, 

the regearcher vho interprets and names components findi that his acorea, 

H or Kj have a completely different meaning than he antloipateB* Any use 

of theae icorea in further r.nalygia i^ill result in a serioua diitortion 

of oonclusionsf Even in Table le, vheri the distortion mlnlmali ve 

find r^ ' to be .78 and one faetor iMdlng of ,23 resulting lis r . and 

ZH 

r of .56 and *38 respeetively* 
oft. 

Conclusioni 

The mathematics and illustratlona lead ua to the eoaclusion that factor 
loadings, used direct ly or as the te^iis of binary valuta are not appropriate 
aa -weighta to produce aomponent eaores from a rotated gQlutlon* But 
intuitlvejiy they raay atill mke senge. In an attempt to diapatch thia idea, 
let ui draw an. anology. Linear composites are alao used aa the basis of 
prediction in multiple regression. However^ no person at all familiar with 
recession ana]^ala would use predtctor-criterion corraljLtionB as regrtssion 
veighti* Why then should the use of correlationa aa weights be conaldtred 
Intuitiviily appealing in component analyals? 
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